Generating retinal flow maps from structural optical coherence tomography with artificial intelligence

Despite​ ​advances​ ​in​ ​artificial​ ​intelligence​ ​(AI),​ ​its​ ​application​ ​in​ ​medical​ ​imaging​ ​has​ ​been burdened​ ​and​ ​limited​ ​by​ ​expert-generated​ ​labels.​ ​We​ ​used​ ​images​ ​from​ ​optical​ ​coherence tomography​ ​angiography​ ​(OCTA),​ ​a​ ​relatively​ ​new​ ​imaging​ ​modality​ ​that​ ​measures​ ​retinal​ ​blood flow,​ ​to​ ​train​ ​an​ ​AI​ ​algorithm​ ​to​ ​generate​ ​flow​ ​maps​ ​from​ ​standard​ ​optical​ ​coherence tomography​ ​(OCT)​ ​images,​ ​exceeding​ ​the​ ​ability​ ​and​ ​bypassing​ ​the​ ​need​ ​for​ ​expert​ ​labeling. Deep​ ​learning​ ​was​ ​able​ ​to​ ​infer​ ​flow​ ​from​ ​single​ ​structural​ ​OCT​ ​images​ ​with​ ​similar​ ​fidelity​ ​to OCTA​ ​and​ ​significantly​ ​better​ ​than​ ​expert​ ​clinicians​ ​(P​ ​<​ ​0.00001).​ ​Our​ ​model​ ​allows​ ​generating flow​ ​maps​ ​from​ ​large​ ​volumes​ ​of​ ​previously​ ​collected​ ​OCT​ ​data​ ​in​ ​existing​ ​clinical​ ​trials​ ​and clinical​ ​practice.​ ​This​ ​finding​ ​demonstrates​ ​a​ ​novel​ ​application​ ​of​ ​AI​ ​to​ ​medical​ ​imaging, whereby​ ​subtle​ ​regularities​ ​between​ ​different​ ​modalities​ ​are​ ​used​ ​to​ ​image​ ​the​ ​same​ ​body​ ​part and​ ​AI​ ​is​ ​used​ ​to​ ​generate​ ​detailed​ ​inferences​ ​of​ ​tissue​ ​function​ ​from​ ​structure​ ​imaging. INTRODUCTION Advances​ ​in​ ​image​ ​analysis​ ​and​ ​artificial​ ​intelligence​ ​(AI)​ ​have​ ​made​ ​computer-aided​ ​diagnosis (CAD)​ ​widely​ ​applicable.​ ​However,​ ​supervised​ ​machine​ ​learning​ ​usually​ ​requires​ ​a​ ​large number​ ​of​ ​expert-defined​ ​labels,​ ​with​ ​two​ ​main​ ​limitations.​ (Valizadegan,​ ​Nguyen,​ ​and Hauskrecht​ ​2013)​​ ​First,​ ​most​ ​of​ ​these​ ​labels​ ​are​ ​manually​ ​generated​ ​by​ ​clinicians,​ ​which​ ​is​ ​a cumbersome,​ ​time-consuming,​ ​and​ ​consequently​ ​costly​ ​process.​ ​Second,​ ​the​ ​use​ ​of​ ​human generated​ ​annotations​ ​as​ ​the​ ​ground​ ​truth​ ​limits​ ​the​ ​learning​ ​ability​ ​of​ ​the​ ​AI,​ ​given​ ​that​ ​it​ ​is problematic​ ​for​ ​AI​ ​to​ ​surpass​ ​the​ ​accuracy​ ​of​ ​humans,​ ​by​ ​definition.​ ​In​ ​addition, expert-generated​ ​labels​ ​suffer​ ​from​ ​inherent​ ​inter-rater​ ​variability,​ ​thereby​ ​limiting​ ​the​ ​accuracy of​ ​the​ ​AI​ ​to​ ​at​ ​most​ ​variable​ ​human​ ​discriminative​ ​abilities.​ ​Thus,​ ​the​ ​use​ ​of​ ​more​ ​accurate, objectively-generated​ ​annotations​ ​would​ ​be​ ​a​ ​key​ ​advance​ ​in​ ​machine​ ​learning​ ​algorithms​ ​in diverse​ ​areas​ ​of​ ​medicine. Optical​ ​coherence​ ​tomography​ ​(OCT)​ ​is​ ​a​ ​non-invasive​ ​imaging​ ​modality​ ​of​ ​structural​ ​retina​ ​​in vivo​.​ ​Since​ ​its​ ​development​ ​in​ ​1991,​ ​OCT​ ​has​ ​become​ ​essential​ ​in​ ​diagnosing​ ​and​ ​assessing most​ ​vision-threatening​ ​conditions​ ​in​ ​ophthalmology.​(Murthy​ ​et​ ​al.​ ​2016)​​ ​A​ ​recent​ ​advance​ ​in OCT​ ​technology​ ​led​ ​to​ ​its​ ​counterpart,​ ​OCT​ ​angiography​ ​(OCTA),​ ​which​ ​measures​ ​blood​ ​flow in​ ​retinal​ ​microvasculature​ ​by​ ​obtaining​ ​repeated​ ​measurements​ ​of​ ​phase​ ​and​ ​intensity​ ​at​ ​the same​ ​scanning​ ​position.​(Wang​ ​et​ ​al.​ ​2007;​ ​de​ ​Carlo​ ​et​ ​al.​ ​2015)​​ ​While​ ​OCTA​ ​can​ ​theoretically be​ ​obtained​ ​using​ ​the​ ​same​ ​OCT​ ​hardware,​ ​in​ ​practice,​ ​OCTA​ ​requires​ ​both​ ​hardware​ ​and software​ ​modifications​ ​to​ ​existing​ ​OCT​ ​machines.​ ​OCTA​ ​can​ ​visualize​ ​both​ ​superficial​ ​and​ ​deep capillary​ ​plexus​ ​of​ ​the​ ​retinal​ ​vasculature​ ​without​ ​an​ ​exogenous​ ​dye,​ ​unlike​ ​fluorescein angiography,​ ​enabling​ ​better​ ​detection​ ​of​ ​overall​ ​retinal​ ​flow​ ​without​ ​potential​ ​side​ ​effects.​ (Ting et​ ​al.​ ​2017)​​ ​Despite​ ​the​ ​advantages,​ ​the​ ​use​ ​of​ ​OCTA​ ​is​ ​not​ ​as​ ​widespread​ ​as​ ​OCT,​ ​due​ ​to​ ​its cost​ ​and​ ​limited​ ​field​ ​of​ ​view​ ​(FOV)​ ​on​ ​currently​ ​commercially​ ​available​ ​devices,​ ​which decreases​ ​the​ ​ability​ ​to​ ​assess​ ​microvascular​ ​complications​ ​of​ ​retinal​ ​vascular​ ​diseases.​ ​In addition,​ ​OCTA​ ​requires​ ​multiple​ ​acquisitions​ ​in​ ​the​ ​same​ ​anatomic​ ​location,​ ​limiting​ ​the​ ​ability to​ ​acquire​ ​interpretable​ ​images​ ​in​ ​eyes​ ​with​ ​unstable​ ​visual​ ​fixation​ ​and​ ​motion​ ​artifacts​ ​from microsaccades.​(Q.​ ​Zhang​ ​et​ ​al.​ ​2016) Given​ ​the​ ​relationship​ ​of​ ​OCT​ ​and​ ​OCTA,​ ​we​ ​sought​ ​to​ ​explore​ ​the​ ​deep​ ​learning’s​ ​ability​ ​to​ ​first infer​ ​between​ ​structure​ ​and​ ​function,​ ​then​ ​generate​ ​an​ ​OCTA-like​ ​en-face​ ​image​ ​from​ ​structural OCT​ ​image​ ​alone.​ ​Human​ ​graders​ ​struggle​ ​to​ ​identify​ ​all​ ​but​ ​the​ ​largest​ ​vessels​ ​on​ ​structural OCT​ ​scans.​ ​A​ ​successful​ ​model​ ​would​ ​result​ ​in​ ​the​ ​acquisition​ ​of​ ​new​ ​information​ ​from preexisting​ ​databases​ ​given​ ​the​ ​ubiquitous​ ​use​ ​of​ ​OCT​ ​and​ ​may​ ​result​ ​in​ ​en-face​ ​images significantly​ ​less​ ​affected​ ​by​ ​artifacts.​ ​Unlike​ ​current​ ​AI​ ​models​ ​which​ ​are​ ​primarily​ ​targeted towards​ ​classification​ ​or​ ​segmentation​ ​of​ ​images,​ ​to​ ​our​ ​knowledge,​ ​this​ ​is​ ​the​ ​first​ ​application​ ​of artificial​ ​neural​ ​networks​ ​to​ ​generate​ ​a​ ​new​ ​image​ ​based​ ​on​ ​a​ ​different​ ​imaging​ ​modality​ ​data.​ ​In addition,​ ​this​ ​is​ ​the​ ​first​ ​example​ ​in​ ​medical​ ​imaging​ ​to​ ​our​ ​knowledge​ ​where​ ​expert​ ​annotations for​ ​training​ ​deep​ ​learning​ ​models​ ​are​ ​bypassed​ ​by​ ​using​ ​objective,​ ​physiologic​ ​measurements. RESULTS Four​ ​different​ ​model​ ​archetypes​ ​(Figure​ ​1A)​ ​were​ ​designed​ ​to​ ​take​ ​a​ ​single​ ​individual​ ​structural B​ ​scan​ ​image​ ​as​ ​input​ ​​ ​and​ ​provide​ ​an​ ​inferred​ ​flow​ ​B​ ​scan​ ​image​ ​as​ ​output​ ​which​ ​included​ ​5 blocks​ ​of​ ​max​ ​pooling​ ​and​ ​upsampling​ ​with​ ​5​ ​convolutional​ ​filters,​ ​5​ ​blocks​ ​with​ ​10​ ​convolutional filters,​ ​9​ ​blocks​ ​with​ ​9​ ​convolutional​ ​filters,​ ​and​ ​9​ ​blocks​ ​with​ ​18​ ​convolutional​ ​filters.​ ​In​ ​addition for​ ​each​ ​set​ ​of​ ​the​ ​4​ ​models,​ ​3​ ​different​ ​bridge​ ​connections​ ​were​ ​tested:​ ​no​ ​bridge​ ​(similar​ ​to traditional​ ​convolutional​ ​autoencoder​ ​network),​ ​element-wise​ ​summation,​ ​and​ ​copy​ ​+ concatenation.​ ​Each​ ​of​ ​these​ ​models​ ​were​ ​trained​ ​from​ ​random​ ​initialization​ ​with​ ​the​ ​same batch​ ​number,​ ​training/validation​ ​datasets,​ ​optimizer,​ ​and​ ​learning​ ​rate​ ​for​ ​5,000​ ​iterations (Figure​ ​1B)​ ​and​ ​the​ ​deepest​ ​model​ ​with​ ​18​ ​convolution​ ​filters​ ​and​ ​with​ ​copy​ ​+​ ​concatenation bridges​ ​had​ ​the​ ​lowest​ ​MSE​ ​(Figure​ ​1C).​ ​The​ ​final​ ​model​ ​had​ ​a​ ​total​ ​of​ ​7.85​ ​million​ ​trainable parameters​ ​and​ ​has​ ​a​ ​space​ ​complexity​ ​of​ ​90​ ​megabytes.​ ​The​ ​model​ ​received​ ​no​ ​additional information​ ​regarding​ ​the​ ​neighboring​ ​slices.​ ​An​ ​extended​ ​training​ ​session​ ​was​ ​performed​ ​with dropout​ ​layers​ ​for​ ​regularization​ ​(Figure​ ​1D).​ ​After​ ​collecting​ ​independently​ ​inferred​ ​flow​ ​B​ ​scan images,​ ​an​ ​en-face​ ​projection​ ​image​ ​was​ ​created​ ​using​ ​the​ ​same​ ​techniques​ ​as​ ​OCTA. A​ ​total​ ​of​ ​401,098​ ​cross-sectional​ ​structural,​ ​macular​ ​spectral-domain​ ​(SD)​ ​OCT​ ​images​ ​from 873​ ​volumes​ ​were​ ​designated​ ​as​ ​a​ ​training​ ​set:​ ​these​ ​were​ ​presented​ ​to​ ​the​ ​deep​ ​learning model,​ ​and​ ​the​ ​output​ ​was​ ​compared​ ​to​ ​the​ ​corresponding​ ​retina-segmented​ ​OCTA​ ​image. Another​ ​76,928​ ​OCT​ ​images​ ​from​ ​independent​ ​171​ ​cubes​ ​were​ ​used​ ​for​ ​cross-validation against​ ​OCTA​ ​images.​ ​A​ ​held​ ​out​ ​test​ ​set​ ​of​ ​92,606​ ​images​ ​of​ ​202​ ​cubes​ ​from​ ​a​ ​different​ ​set​ ​of patients​ ​was​ ​then​ ​used​ ​for​ ​comparison​ ​of​ ​deep​ ​learning​ ​model​ ​performance​ ​against​ ​the​ ​OCTA images. The​ ​model​ ​was​ ​trained​ ​with​ ​60,000​ ​iterations.​ ​The​ ​learning​ ​curve​ ​with​ ​the​ ​mean​ ​squared​ ​error (MSE)​ ​of​ ​the​ ​training​ ​iterations​ ​and​ ​the​ ​validation​ ​set​ ​are​ ​shown​ ​in​ ​Figure​ ​1B.​ ​The​ ​model achieved​ ​a​ ​minimal​ ​cross-validation​ ​MSE​ ​of​ ​9.9482​ ​x​ ​10​ .​ ​The​ ​weights​ ​that​ ​produced​ ​the​ ​best cross-validation​ ​MSE​ ​were​ ​then​ ​used​ ​for​ ​comparison​ ​against​ ​OCTA.​ ​The​ ​performance​ ​on​ ​the held-out​ ​test​ ​set​ ​achieved​ ​an​ ​MSE​ ​of​ ​7.7665​ ​x​ ​10​ .​ ​The​ ​fidelity​ ​by​ ​peak​ ​signal​ ​to​ ​noise​ ​ratio​ ​was 31.10​ ​db.​ ​​ ​Figure​ ​2​ ​shows​ ​examples​ ​of​ ​deep​ ​learning​ ​inference​ ​of​ ​retinal​ ​flow​ ​from cross-sectional​ ​structural​ ​OCT​ ​images​ ​of​ ​the​ ​held​ ​out​ ​test​ ​set​ ​compared​ ​to​ ​the​ ​corresponding OCTA​ ​images.​ ​From​ ​structural​ ​OCT​ ​images,​ ​the​ ​deep​ ​learning​ ​model​ ​was​ ​able​ ​to​ ​identify​ ​both the​ ​large​ ​and​ ​medium-sized​ ​retinal​ ​vessels​ ​as​ ​well​ ​as​ ​the​ ​retinal​ ​microvasculature​ ​at​ ​a​ ​level​ ​of detail​ ​similar​ ​to​ ​the​ ​OCTA​ ​image.​ ​In​ ​addition,​ ​the​ ​model​ ​identified​ ​small​ ​vessels​ ​that​ ​are​ ​not apparent​ ​on​ ​the​ ​structural​ ​OCT​ ​image.​ ​Furthermore,​ ​the​ ​model​ ​was​ ​able​ ​to​ ​learn​ ​the segmentation​ ​of​ ​the​ ​retina​ ​and​ ​isolate​ ​structural​ ​features​ ​of​ ​the​ ​retina.​ ​Given​ ​that​ ​the​ ​model takes​ ​as​ ​input​ ​only​ ​a​ ​single​ ​structural​ ​B​ ​scan​ ​image,​ ​we​ ​asked​ ​three​ ​masked​ ​independent​ ​retina specialty​ ​trained​ ​experts​ ​to​ ​identify​ ​vessels​ ​on​ ​a​ ​single​ ​structural​ ​B​ ​scan​ ​image.​ ​Comparison​ ​of model​ ​output​ ​to​ ​three​ ​different​ ​masked​ ​clinicians​ ​revealed​ ​that​ ​the​ ​trained​ ​model​ ​was​ ​able​ ​to significantly​ ​outperform​ ​clinicians​ ​in​ ​terms​ ​of​ ​specificity,​ ​positive​ ​predictive​ ​value,​ ​and​ ​negative predictive​ ​value​ ​when​ ​using​ ​OCTA​ ​as​ ​ground​ ​truth. To​ ​examine​ ​the​ ​performance​ ​of​ ​the​ ​model​ ​on​ ​other​ ​retinal​ ​pathologies​ ​outside​ ​of​ ​what​ ​the algorithm​ ​was​ ​trained,​ ​the​ ​weights​ ​from​ ​the​ ​lowest​ ​cross​ ​validation​ ​MSE​ ​were​ ​used​ ​to​ ​infer​ ​flow of​ ​each​ ​cross-sectional​ ​structural​ ​OCT​ ​image​ ​in​ ​a​ ​volume.​ ​Average​ ​projection​ ​of​ ​resulting inferred​ ​flow​ ​volume​ ​was​ ​used​ ​to​ ​create​ ​en-face​ ​projection​ ​maps​ ​of​ ​flow​ ​(Figure​ ​3). Surprisingly,​ ​even​ ​without​ ​three-dimensional​ ​knowledge​ ​of​ ​the​ ​location​ ​of​ ​vessels​ ​nor knowledge​ ​of​ ​the​ ​neighboring​ ​cross​ ​sectional​ ​slice,​ ​the​ ​inferred​ ​flow​ ​by​ ​deep​ ​learning​ ​generated contiguous​ ​vessel​ ​maps​ ​similar​ ​to​ ​OCTA. Compared​ ​to​ ​en-face​ ​projections​ ​of​ ​the​ ​structural​ ​OCT​ ​volumes​ ​(Figure​ ​3​ ​A,​ ​D,​ ​G,​ ​J),​ ​the​ ​AI generated​ ​flow​ ​maps​ ​(Figure​ ​3​ ​B,​ ​E,​ ​H,​ ​K)​ ​show​ ​more​ ​detail​ ​of​ ​the​ ​superficial​ ​retinal vasculature.​ ​The​ ​map​ ​of​ ​retinal​ ​vasculature​ ​generated​ ​by​ ​the​ ​model​ ​was​ ​much​ ​more​ ​detailed​ ​at the​ ​superficial​ ​retina​ ​than​ ​deeper​ ​in​ ​the​ ​retina​ ​and​ ​was​ ​superior​ ​to​ ​structural​ ​OCT​ ​en-face projections.​ ​This​ ​discrepancy​ ​was​ ​easily​ ​demonstrated​ ​in​ ​the​ ​pathologic​ ​eyes,​ ​(Figure​ ​3​ ​E,​ ​H,​ ​K) in​ ​which​ ​the​ ​superficial​ ​capillary​ ​plexus​ ​were​ ​affected​ ​by​ ​ischemia​ ​more​ ​than​ ​deep​ ​capillary plexus.​ ​In​ ​eyes​ ​with​ ​diabetic​ ​ischemia​ ​(Figure​ ​3​ ​E,​ ​F)​ ​and​ ​branch​ ​retinal​ ​vascular​ ​occlusion (Figure​ ​3​ ​H,I),​ ​OCTA​ ​reveals​ ​a​ ​higher​ ​density​ ​of​ ​deep​ ​capillary​ ​plexus​ ​than​ ​our​ ​model’s​ ​flow output.​ ​Intact​ ​cilioretinal​ ​artery​ ​in​ ​the​ ​setting​ ​of​ ​central​ ​retinal​ ​artery​ ​occlusion​ ​(CRAO)​ ​preserves the​ ​superficial​ ​capillary​ ​plexus​ ​in​ ​the​ ​area​ ​perfused​ ​by​ ​cilioretinal​ ​artery.​ ​As​ ​expected,​ ​our model’s​ ​results​ ​were​ ​comparable​ ​to​ ​OCTA​ ​in​ ​the​ ​area​ ​perfused​ ​by​ ​cilioretinal​ ​artery​ ​but​ ​not​ ​able to​ ​show​ ​the​ ​remaining​ ​deep​ ​capillary​ ​plexus​ ​elsewhere​ ​in​ ​the​ ​macula.​ ​(Figure​ ​3​ ​K,​ ​L). When​ ​compared​ ​to​ ​color​ ​fundus​ ​photography​ ​and​ ​en-face​ ​projections​ ​of​ ​structural​ ​OCT volumes,​ ​the​ ​retinal​ ​vasculature​ ​map​ ​generated​ ​by​ ​the​ ​model​ ​was​ ​better​ ​for​ ​visualizing​ ​the superficial​ ​capillary​ ​networks.​ ​As​ ​shown​ ​in​ ​Figure​ ​4,​ ​the​ ​deep​ ​learning​ ​image​ ​highlights​ ​the​ ​area of​ ​capillary​ ​dropout​ ​and​ ​intact​ ​flow.​ ​Both​ ​superficial​ ​arterioles​ ​and​ ​superficial​ ​capillary​ ​networks are​ ​clearly​ ​visible​ ​in​ ​the​ ​area​ ​supplied​ ​by​ ​cilioretinal​ ​artery​ ​on​ ​deep​ ​learning​ ​image.​ ​In​ ​contrast, only​ ​larger​ ​retinal​ ​vessels​ ​are​ ​visible,​ ​and​ ​the​ ​integrities​ ​of​ ​capillary​ ​plexus​ ​are​ ​difficult​ ​to​ ​assess on​ ​the​ ​color​ ​photo​ ​and​ ​structural​ ​OCT​ ​en-face​ ​projections.​ ​Similarly,​ ​Figure​ ​5​ ​shows​ ​that,​ ​for​ ​a normal​ ​eye,​ ​the​ ​retinal​ ​vasculature​ ​map​ ​generated​ ​by​ ​the​ ​model​ ​demonstrates​ ​the​ ​superficial capillary​ ​networks​ ​with​ ​superior​ ​detail​ ​compared​ ​to​ ​both​ ​the​ ​corresponding​ ​color​ ​fundus photograph​ ​and​ ​the​ ​late​ ​phase​ ​fluorescein​ ​angiogram.​ ​OCTA​ ​continues​ ​to​ ​highlight​ ​the​ ​retinal microvasculature​ ​with​ ​the​ ​highest​ ​detail.​ ​​ ​Manual​ ​segmentation​ ​of​ ​vessels​ ​on​ ​all​ ​four​ ​imaging modalities​ ​was​ ​performed​ ​(Figure​ ​6).​ ​For​ ​second​ ​order​ ​vessels,​ ​87.5%,​ ​97.5%,​ ​and​ ​100%​ ​of​ ​the vessels​ ​were​ ​identified​ ​by​ ​color,​ ​FA,​ ​and​ ​deep​ ​learning​ ​generated​ ​flow​ ​maps.​ ​Deep​ ​learning was​ ​able​ ​to​ ​identify​ ​significantly​ ​more​ ​third​ ​order​ ​vessels​ ​compared​ ​to​ ​color​ ​images​ ​(p​ ​=​ ​0.0320) and​ ​was​ ​able​ ​to​ ​identify​ ​significantly​ ​more​ ​fourth​ ​order​ ​vessels​ ​compared​ ​to​ ​both​ ​color​ ​and​ ​FA images​ ​(p​ ​=​ ​1.86​ ​x​ ​10​​ ​and​ ​5.01​ ​x​ ​10​​ ​respectively). DISCUSSION Our​ ​study​ ​demonstrates​ ​that​ ​a​ ​deep​ ​learning​ ​model​ ​can​ ​be​ ​trained​ ​to​ ​recognize​ ​features​ ​of​ ​OCT images​ ​that​ ​allow​ ​successful​ ​identification​ ​of​ ​retinal​ ​vasculature​ ​on​ ​cross​ ​sectional​ ​OCT​ ​scans in​ ​a​ ​fully​ ​automated​ ​fashion​ ​and​ ​generate​ ​en-face​ ​projection​ ​flow​ ​maps.​ ​The​ ​deep​ ​learning model​ ​identified​ ​both​ ​the​ ​retinal​ ​vessels​ ​that​ ​were​ ​easily​ ​seen​ ​on​ ​structural​ ​OCT​ ​b-scan​ ​images as​ ​well​ ​as​ ​the​ ​retinal​ ​microvasculature​ ​that​ ​was​ ​not​ ​apparent​ ​to​ ​the​ ​clinician​ ​on​ ​standard​ ​OCT (Figure​ ​2)​ ​and​ ​showed​ ​significantly​ ​more​ ​retinal​ ​vessels​ ​compared​ ​to​ ​structural​ ​OCT​ ​en-face projections,​ ​color​ ​and​ ​FA​ ​images.​ ​Surprisingly​ ​deep​ ​learning​ ​was​ ​able​ ​to​ ​generate​ ​detailed​ ​flow maps​ ​of​ ​the​ ​retinal​ ​vessels​ ​in​ ​a​ ​variety​ ​of​ ​retinal​ ​conditions​ ​using​ ​standard,​ ​ubiquitously available​ ​structural​ ​imaging. Taking​ ​advantage​ ​of​ ​an​ ​already​ ​existing​ ​imaging​ ​modality​ ​as​ ​the​ ​ground​ ​truth,​ ​our​ ​study bypasses​ ​the​ ​need​ ​to​ ​generate​ ​expert​ ​annotations​ ​entirely.​ ​Furthermore,​ ​acquiring​ ​a​ ​retinal vascular​ ​map​ ​from​ ​OCT​ ​via​ ​deep​ ​learning​ ​enables​ ​the​ ​comparison​ ​of​ ​retinal​ ​vascular​ ​structure versus​ ​function​ ​of​ ​retinal​ ​vasculature​ ​using​ ​OCT​ ​and​ ​OCTA,​ ​respectively.​ ​One​ ​of​ ​the​ ​potential mechanisms​ ​by​ ​which​ ​deep​ ​learning​ ​infers​ ​flow​ ​from​ ​structural​ ​images​ ​may​ ​be​ ​similar​ ​to speckle-variance​ ​(SV)​ ​processing​ ​method.​ ​​ ​SV​ ​imaging​ ​measures​ ​decorrelation​ ​between​ ​the OCT​ ​signals​ ​that​ ​are​ ​generated​ ​by​ ​speckle​ ​or​ ​backscattered​ ​light​ ​from​ ​biological tissues.​(Mahmud​ ​et​ ​al.​ ​2013;​ ​A.​ ​Zhang​ ​et​ ​al.​ ​2015)​​ ​Different​ ​speckle​ ​patterns​ ​are​ ​created​ ​due​ ​to moving​ ​particles​ ​in​ ​biological​ ​tissues​ ​such​ ​as​ ​red​ ​blood​ ​cells,​ ​thus​ ​enabling​ ​measuring​ ​flow. However​ ​SV​ ​imaging​ ​requires​ ​repeated​ ​imaging​ ​in​ ​the​ ​same​ ​anatomic​ ​location,​ ​whereas​ ​the algorithm​ ​presented​ ​here​ ​is​ ​able​ ​to​ ​infer​ ​flow​ ​from​ ​a​ ​single​ ​structural​ ​OCT​ ​scan.​ ​Thus,​ ​deep learning​ ​may​ ​be​ ​detecting​ ​the​ ​likely​ ​decorrelation​ ​that​ ​exists​ ​on​ ​a​ ​single​ ​scan​ ​based​ ​on​ ​trained relationship​ ​between​ ​OCT​ ​and​ ​OCTA. The​ ​novel​ ​application​ ​of​ ​deep​ ​learning​ ​in​ ​our​ ​study​ ​infers​ ​flow​ ​from​ ​traditional​ ​OCT​ ​images.​ ​This finding​ ​has​ ​significant​ ​clinical​ ​applications.​ ​First,​ ​OCT​ ​is​ ​the​ ​most​ ​commonly​ ​performed​ ​eye procedure,​ ​thus​ ​resulting​ ​in​ ​extensive​ ​OCT​ ​databases​ ​in​ ​most​ ​clinics​ ​(including​ ​those​ ​acquired before​ ​OCTA​ ​was​ ​available).​ ​This​ ​large​ ​imaging​ ​cohort​ ​may​ ​allow​ ​us​ ​to​ ​determine​ ​the​ ​natural history​ ​of​ ​vascular​ ​changes,​ ​blood​ ​flow,​ ​and​ ​clinical​ ​outcomes​ ​in​ ​retinal​ ​diseases,​ ​similar​ ​to previous​ ​studies​ ​involving​ ​fundus​ ​photographs​(R.​ ​Klein​ ​et​ ​al.​ ​2004,​ ​2007)​​ ​but​ ​allowing​ ​much precise​ ​3D​ ​volumetric​ ​information​ ​of​ ​retinal​ ​vasculature​ ​and​ ​thickness​ ​of​ ​retinal​ ​layers.​ ​Second, we​ ​do​ ​not​ ​know​ ​whether​ ​the​ ​output​ ​of​ ​our​ ​deep​ ​learning​ ​algorithm​ ​is​ ​structural​ ​or​ ​functional information​ ​given​ ​that​ ​the​ ​input​ ​was​ ​strictly​ ​based​ ​on​ ​structural​ ​images​ ​but​ ​the​ ​training​ ​was performed​ ​with​ ​functional​ ​data​ ​(OCTA).​ ​If​ ​the​ ​algorithm​ ​is​ ​indeed​ ​providing​ ​only​ ​structural​ ​map of​ ​the​ ​vasculature,​ ​then​ ​the​ ​results​ ​may​ ​allow​ ​us​ ​to​ ​compare​ ​the​ ​discrepancy​ ​between​ ​structure and​ ​function​ ​when​ ​we​ ​compare​ ​them​ ​to​ ​OCTA​ ​images.​ ​With​ ​longitudinal​ ​data,​ ​we​ ​may​ ​discern when​ ​the​ ​structural​ ​changes​ ​occurs:​ ​prior​ ​to​ ​the​ ​decline​ ​in​ ​function​ ​or​ ​vice​ ​versa.​ ​Future comparisons​ ​of​ ​the​ ​OCTA​ ​and​ ​deep​ ​learning​ ​images​ ​of​ ​the​ ​eyes​ ​that​ ​underwent​ ​a​ ​recent vascular​ ​insult,​ ​in​ ​which​ ​a​ ​clear​ ​difference​ ​exists​ ​between​ ​structure​ ​and​ ​function​ ​would​ ​be useful.​ ​More​ ​research​ ​is​ ​needed​ ​to​ ​establish​ ​the​ ​utility​ ​of​ ​applying​ ​deep​ ​learning​ ​to​ ​structurally correlated​ ​images,​ ​but​ ​a​ ​similar​ ​principle​ ​could​ ​be​ ​applied​ ​in​ ​different​ ​fields​ ​of​ ​medicine​ ​where structural​ ​imaging​ ​is​ ​routinely​ ​obtained​ ​and​ ​functional​ ​imaging​ ​data​ ​is​ ​available​ ​for​ ​use​ ​of​ ​the ground​ ​truth,​ ​such​ ​as​ ​computerized​ ​tomography​ ​or​ ​magnetic​ ​resonance​ ​imaging. In​ ​comparison​ ​to​ ​structural​ ​OCT​ ​en-face​ ​projections,​ ​we​ ​show​ ​that​ ​the​ ​deep​ ​learning​ ​inferred flow​ ​maps​ ​are​ ​able​ ​to​ ​provide​ ​better​ ​definition​ ​of​ ​retinal​ ​vessels​ ​(Figures​ ​3,​ ​4).​ ​Powner​ ​et​ ​al.​ ​has previously​ ​shown​ ​that​ ​en-face​ ​projections​ ​of​ ​the​ ​structural​ ​OCT​ ​volumes​ ​leave​ ​behind​ ​basement membranes​ ​which​ ​are​ ​unperfused​ ​but​ ​will​ ​appear​ ​hyperreflective​ ​on​ ​the​ ​structural​ ​OCT imaging.​(Powner​ ​et​ ​al.​ ​2016)​​ ​This​ ​suggests​ ​that​ ​the​ ​AI​ ​generated​ ​flow​ ​maps​ ​may​ ​have​ ​more clinical​ ​utility​ ​than​ ​en-face​ ​projections​ ​of​ ​structural​ ​OCT​ ​volumes​ ​since​ ​the​ ​latter​ ​can​ ​not distinguish​ ​between​ ​perfused​ ​and​ ​unperfused​ ​vessels. In​ ​our​ ​work,​ ​we​ ​utilized​ ​a​ ​U​ ​shaped​ ​autoencoder​ ​network​ ​as​ ​the​ ​final​ ​model​ ​that​ ​has​ ​traditionally been​ ​used​ ​for​ ​semantic​ ​medical​ ​segmentation.​ (Ronneberger,​ ​Fischer,​ ​and​ ​Brox​ ​2015)​​ ​This deep​ ​neural​ ​network​ ​architecture​ ​uses​ ​bridges​ ​to​ ​maintain​ ​high​ ​resolution​ ​spatial​ ​information that​ ​is​ ​normally​ ​lost​ ​during​ ​pooling​ ​operations.​ ​In​ ​addition,​ ​deeper​ ​neural​ ​networks​ ​have generally​ ​been​ ​found​ ​to​ ​improve​ ​performance​ ​which​ ​has​ ​been​ ​shown​ ​in​ ​the​ ​computer​ ​vision research​ ​with​ ​improved​ ​accuracy​ ​with​ ​ImageNet​ ​image​ ​classification​ ​as​ ​networks​ ​became deeper.​(Simonyan​ ​and​ ​Zisserman​ ​2014;​ ​He​ ​et​ ​al.​ ​2015;​ ​Szegedy​ ​et​ ​al.​ ​2015)​ ​ ​With​ ​our​ ​data,​ ​we have​ ​empirically​ ​found​ ​that​ ​the​ ​copy​ ​and​ ​concatenation​ ​bridge​ ​with​ ​deeper​ ​number​ ​of convolutional​ ​layers​ ​led​ ​to​ ​the​ ​best​ ​performance​ ​compared​ ​to​ ​models​ ​with​ ​shallower​ ​networks, no​ ​bridge​ ​connections,​ ​and​ ​element-wise​ ​summation​ ​bridge​ ​connections.​ ​Future​ ​work​ ​could include​ ​further​ ​optimization​ ​and​ ​hyperparameter​ ​model​ ​selection​ ​and​ ​comparison​ ​to​ ​other architectures​ ​such​ ​as​ ​V-net,​(Milletari,​ ​Navab,​ ​and​ ​Ahmadi​ ​2016)​​ ​recurrent​ ​convolutional layers,​(Liang​ ​and​ ​Hu​ ​2015)​​ ​two-pathway​ ​convolutional​ ​networks,​(Havaei​ ​et​ ​al.​ ​2017)​​ ​and​ ​hybrid models​ ​with​ ​convolutional​ ​layers​ ​with​ ​recurrent​ ​network​ ​layers. More​ ​broadly,​ ​a​ ​similar​ ​approach​ ​could​ ​have​ ​application​ ​in​ ​many​ ​other​ ​imaging​ ​modalities, where​ ​the​ ​same​ ​object​ ​is​ ​imaged​ ​with​ ​sensitivity​ ​to​ ​different​ ​properties​ ​of​ ​the​ ​object​ ​that​ ​is​ ​being imaged.​ ​For​ ​example,​ ​radiological/MRI​ ​measurements​ ​of​ ​the​ ​same​ ​body​ ​part​ ​are​ ​routinely conducted​ ​with​ ​different​ ​contrasts,​ ​taking​ ​advantage​ ​of​ ​the​ ​sensitivity​ ​of​ ​different​ ​contrasts​ ​to different​ ​properties​ ​of​ ​the​ ​tissue.​ ​While​ ​this​ ​allows​ ​measurements​ ​of​ ​different​ ​tissue​ ​properties,​ ​it is​ ​also​ ​time-consuming.​ ​In​ ​some​ ​cases,​ ​measurements​ ​that​ ​are​ ​highly​ ​sensitive​ ​to​ ​certain​ ​tissue properties​ ​may​ ​require​ ​invasive​ ​injection​ ​of​ ​contrast​ ​agents,​ ​or​ ​exposure​ ​to​ ​x-ray​ ​radiation.​ ​The present​ ​results​ ​demonstrate​ ​that​ ​even​ ​a​ ​very​ ​small​ ​amount​ ​of​ ​sensitivity​ ​to​ ​variations​ ​in​ ​a​ ​tissue property​ ​may​ ​be​ ​enough​ ​for​ ​a​ ​deep​ ​neural​ ​network​ ​algorithm​ ​to​ ​detect​ ​variations​ ​in​ ​the dependent​ ​image​ ​properties,​ ​enabling​ ​accurate​ ​inference​ ​of​ ​the​ ​physiological​ ​features​ ​of​ ​the imaged​ ​body​ ​part,​ ​even​ ​when​ ​these​ ​features​ ​are​ ​not​ ​readily​ ​visible​ ​to​ ​an​ ​expert,​ ​and undetectable​ ​by​ ​means​ ​of​ ​other​ ​image​ ​processing​ ​algorithms.​ ​For​ ​example,​ ​MR​ ​angiography (MRA)​ ​and​ ​anatomical​ ​MRI​ ​are​ ​often​ ​both​ ​required​ ​for​ ​imaging​ ​of​ ​soft​ ​tissue​ ​and​ ​imaging​ ​of blood​ ​vessels​ ​in​ ​the​ ​same​ ​organ.​ ​If​ ​the​ ​anatomical​ ​MRI​ ​possesses​ ​subtle​ ​sensitivity​ ​to​ ​the structure​ ​of​ ​the​ ​blood​ ​vessels,​ ​as​ ​demonstrated​ ​in​ ​the​ ​present​ ​study​ ​for​ ​OCT,​ ​it​ ​is​ ​possible​ ​that information​ ​analogous​ ​that​ ​derived​ ​from​ ​MRA​ ​could​ ​be​ ​inferred​ ​directly​ ​from​ ​an​ ​anatomical​ ​MRI scan. Our​ ​approach​ ​has​ ​a​ ​number​ ​of​ ​limitations.​ ​The​ ​data​ ​collected​ ​for​ ​this​ ​study​ ​was​ ​done​ ​at​ ​a​ ​single academic​ ​center​ ​with​ ​a​ ​device​ ​from​ ​a​ ​single​ ​manufacturer​ ​and​ ​a​ ​consistent​ ​imaging​ ​protocol. While​ ​this​ ​may​ ​limit​ ​the​ ​immediate​ ​generalizability​ ​of​ ​this​ ​method,​ ​the​ ​weights​ ​learned​ ​in​ ​this work​ ​could​ ​be​ ​used​ ​as​ ​the​ ​starting​ ​point​ ​for​ ​transfer​ ​learning,​ (Caruana​ ​1995;​ ​Bengio​ ​and​ ​Others 2012)​​ ​allowing​ ​the​ ​model​ ​to​ ​learn​ ​to​ ​infer​ ​with​ ​images​ ​from​ ​other​ ​devices​ ​rapidly,​ ​and​ ​with substantially​ ​less​ ​data.​ ​Additionally,​ ​future​ ​studies​ ​will​ ​need​ ​to​ ​evaluate​ ​the​ ​clinical​ ​correlations between​ ​our​ ​deep​ ​learning​ ​inferred​ ​flow​ ​maps​ ​and​ ​retinal​ ​perfusion​ ​which​ ​would​ ​include oxygenation​ ​of​ ​vascularized​ ​tissue,​ ​tissue​ ​oxygen​ ​consumption,​ ​and/or​ ​flow​ ​velocity. In​ ​conclusion,​ ​we​ ​show​ ​that​ ​deep​ ​learning​ ​is​ ​able​ ​to​ ​generate​ ​flow​ ​maps​ ​of​ ​superficial​ ​retinal circulation​ ​using​ ​structural​ ​OCT​ ​images​ ​alone.​ ​This​ ​approach​ ​may​ ​be​ ​used​ ​to​ ​analyze​ ​existing OCT​ ​datasets​ ​or​ ​be​ ​integrated​ ​into​ ​existing​ ​OCT​ ​machines​ ​today.​ ​In​ ​addition,​ ​this​ ​methodology of​ ​inferring​ ​weakly​ ​correlated​ ​images​ ​may​ ​be​ ​useful​ ​in​ ​many​ ​other​ ​imaging​ ​applications. AUTHOR​ ​CONTRIBUTIONS Conception​ ​and​ ​design​ ​(CSL,​ ​AYL,​ ​AT);​ ​analysis​ ​and​ ​interpretation​ ​(CSL,​ ​AJT,​ ​YW,​ ​SX,​ ​ASR, AT,​ ​RKW,​ ​AYL);​ ​writing​ ​the​ ​article​ ​(CSL,​ ​AJT,​ ​AYL);​ ​critical​ ​revision​ ​of​ ​the​ ​article​ ​(YW,​ ​SX,​ ​ASR, NPD,​ ​QZ,​ ​AT,​ ​RKW);​ ​final​ ​approval​ ​of​ ​the​ ​article​ ​(CSL,​ ​AJT,​ ​YW,​ ​SX,​ ​ASR,​ ​NPD,​ ​QZ,​ ​AT,​ ​RKW, AYL);​ ​data​ ​collection​ ​(CSL,​ ​QZ,​ ​RKW,​ ​AYL);​ ​statistical​ ​expertise​ ​(CSL,​ ​AYL);​ ​literature​ ​search (CSL,​ ​AJT,​ ​NPD). CONFLICT​ ​OF​ ​INTEREST​ ​DISCLOSURES Ruikang​ ​Wang​ ​received​ ​research​ ​support​ ​from​ ​Carl​ ​Zeiss​ ​Meditec,​ ​Inc.​ ​and​ ​co-owns​ ​a​ ​patent​ ​of OCTA​ ​technology​ ​with​ ​Oregon​ ​Health​ ​&​ ​Science​ ​University.​ ​No​ ​other​ ​conflicts​ ​of​ ​interest​ ​exist for​ ​the​ ​remaining​ ​authors.